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Abstract 

We establish tight bounds on the amount on nonuniformity that is neces- 
sary for extracting a string with randomness rate 1 from a single source of 
randomness with lower randomness rate. More precisely, as instantiations of 
more general results, we show that while 0(1) amount of advice regarding the 
source is not enough for extracting a string with randomness rate 1 from a 
source string with constant subunitary random rate, w(l) amount of advice is. 
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1 Introduction 

By and large, randomness extraction is an algorithmical process that constructs a 
source of randomness of high quality from one or several sources of lower quality. If 
we restrict to the case when there is only one input source, one wants to design an 
effective transformation E from the set of n-bit strings to the set of m-bit strings 
such that for any source x with randomness at least k (where k < n), E(x) has 
randomness ~ m. It is desirable to have m ~ k (i.e., to extract all, or almost 
all, of the randomness in the source). The problem of randomness extraction has 
been modeled in two ways. In the first model (which has been studied extensively 
in computational complexity theory), a source is a probability distribution X over 
{0, l} n and its randomness is given by the min-entropy H^X). In the second 
model, a source is a string x € {0, l} ra , its randomness is given by its Kolmogorov 
complexity C(x), and the algorithmical procedure is called a Kolmogorov extractor. 
In this paper, the focus is on Kolmogorov extractors. 

Thus, Kolmogorov extractors are procedures that increase the Kolmogorov com- 
plexity rate of strings and sequences. Their explicit study was initiated by Fortnow, 
Hitchcock, A. Pavan, Vinodchandran and Wang |FHP + 06 for the case of finite 



strings and by Reimann [Rei04j for the case of infinite strings. The recent pa- 
per |ZimlO| is a survey of this field. 

It is well-known that randomness extraction by uniform procedures from a single 
source is not possible (if we exclude some trivial cases). We investigate the amount 
of non-uniformity that is necessary for Kolmogorov extractors that extract from 
one source. As a consequence of a result of Vereshchagin and Vyugin |VV02| . we 
note that obtaining a source with randomness rate 1 from a source with randomness 
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rate, say, 0.99 is not possible even if the extractor has access to a constant amount of 
non-uniform information. In contrast, we show that an cu(l) amount of non-uniform 
information is sufficient for this task. 

We continue with a more detailed discussion of the two types of results and of 
the technical method that we use. 

Formally, given the parameter k < n, one would like to have a function E : 
{0, l} n {0, l} m such that, whenever C(x) > k, it holds that C(E(x)) « m . It is 
well known that no such computable function E exists for non-trivial parameters. 
Indeed for any given E, consider the string y € {0, l} m with the largest number 
of preimages. Then C{y \ n) = O(l) and among its at least 2 n ~ m preimages there 
must be some x with C(x) > n — m. In other words, for any given E, there are 
some strings (such as the above x) on which E fails. Thus, in order for a function E 
to extract randomness from any source x with randomness > k, E must have some 
additional information a x , which we call advice about the source. The question is 
how much such advice information should be provided. 



Fortnow et al. FHP + 06| have shown that a constant number of advice bits are 



sufficient if one settles to extracting from strings with linear randomness a string 
whose randomness rate is 1 — e. More precisely, they show that for any positive 
rational numbers a and e, there exists a polynomial-time computable function E and 
a constant h such that for any x £ {0, l} n with C(x) > an, there exists a string a x 
of length h such that C(E(x, a x )) > (1 — e)m and m > cn, for some constant c that 
depends on a and e. Note that this result implies that it is possible to construct 
in polynomial time a list with 2 h strings and one of them is guaranteed to have 
Kolmogorov complexity at least (1 — e)m. 

The shortcoming of Fortnow et al's result is that the randomness rate of the 
output is not 1. It would be desirable that C(E(x, a x )) > m — o(m). We first remark 
that, as a consequence of a result of Vereshchagin and Vyugin [VV02| . randomness 
rate 1 cannot be obtained with a constant number of bits of advice about the input. 
Indeed, we show the following. 

Fact 1 If a computable function E : {0, l} n x {0, l} h — > {0, l} m has the property that 
for all strings x with C{x) > an, it holds that there exists a x such that C(E(x, a x )) > 
(1 — e)m, then e > qT^ft^t ~~ (provided that m = w(logn + h)). 

In contrast with the above impossibility result, we show that from sources with 
a linear amount of randomness, one can extract a string with randomness rate 1 
with basically any non-constant amount of advice, such as, for example, the inverse 
of the Ackerman function. This is an instantiation of the following more general 
result. 



Fact 2 (Informal statement; see Theorem \4-4\ f or f u ^ statement.) For any m = 
m{n) computable from n there exists a computable function E with the following 
property: 

For every n-bit string x with complexity > m, there exists a string a x of length 
u>(log^) such that C(E(x,a x )) = m — o(m) and the length of E(x,a x ) is m. 

Note that the function E from Fact [2] is computable, but no complexity bound is 
claimed for it. We can obtain an extractor E computable by a polynomial-size circuit 



The randomness rate of an n-bit string x is C(x)/r 
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with almost all the properties from Theorem [2j Basically the weakening is that the 
output length m has to be < (logn) c , for some positive constant c. Moreover 
the polynomial-size circuit is itself computable, in the sense that there exists an 
algorithm that on input n outputs the description of the circuit that computes 
E. We call such a circuit an effectively constructible circuit. (Note: Without the 
restriction that E is computable, the result would be trivial because a polynomial- 
size circuit computing E could simply have a random string hardwired into it.) 

Fact 3 (Informal statement; see Theorem \4.6\ for full statement.) For every con- 
stant c and any m = m(n) computable in polynomial time from n such that m < 
(logn) c there exists a function E, which is computable by a polynomial- size effec- 
tively constructible circuit with the following property: 

For every n-bit string x with complexity > m, there exists a string a x of length 
w(log^) such that C(E(x,a x )) = m — o(m) and the length of E(x,a x ) is mil 

Discussion of technical aspects. We present the main ideas in the proofs of 
Fact [2j and Fact [3j A Kolmogorov extractor is a computable ensemble of functions 
E : {0, l} ni x {0, l}™ 2 ->• {0, l} m such that for all x € {0, l} ni and y € {0, l}™ 2 
that have Kolmogorov complexity above a certain threshold value and that are 
sufficiently independent (which roughly means that C(y | x) w C(y)), it holds that 
C(E(x,y)) n m. 

To obtain the extractors E that require a small amount of non-uniform infor- 
mation about the source (see Fact [2] and Fact [3|), we need to show that for each 
x, it is enough to have a short string a x such that E(x,a x ) has randomness rate 1 
and contains almost all the randomness of x. The solution is based on the fact that 
from x and a short string that is random even conditioned by x, one can extract 
almost all the randomness of x. This is similar to the well-studied case of seeded 
extractors in computational complexity, with the remark that we can have shorter 
seeds because requiring that the output has randomness rate equal to 1 is weaker 
than requiring that the output is statistically close to the uniform distribution (as 
stipulated in the definition of seeded extractors). Then we take a x to be such a short 
seed. The above fact is obtained via an elementary use of the probabilistic method. 
We first identify a combinatorial object, called a balanced table, that characterizes a 
Kolmogorov extractor, in the sense that the table of a Kolmogorov extractor must 
satisfy the combinatorial constraints of a balanced table. We show (with the proba- 
bilistic method) that such an object exists with a seed of length w(log(n/m)), where 
n is the length of x and m is the Kolmogorov complexity of x. This establishes 
Fact [2j Since the function E from Fact [2] is obtained via the probabilistic method, 
we cannot claim any complexity bound for it. To obtain the Kolmogorov extractor in 
Fact[3j which is computed by polynomial-size circuits, we derandomize the construc- 
tion from Fact [2j using a method of Musatov [MuslO . The key observation is that 
the combinatorial constraints of a balanced table can be checked by constant-depth 
circuits of relatively small size. The argument goes as follows: (a) these constraints 
require that in all sufficiently large rectangles of the table no element appears too 
many times; (b) thus one needs to count the occurrence of each element in every 

In the conference version [Zimllj . it was claimed that this result holds for m(n) < cn, for some 
fixed constant c. The proof contained an error, and we can only prove the result for m(n) < log c (n). 
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sufficiently large rectangle of the table; (c) by a well-known result of Ajtai Ajt93 



this operation can be done with sufficient accuracy by constant-depth circuits with 
relatively small size. Therefore, we can use the Nisan-Wigderson ([NW94 ) pseudo- 
random generator NW-gen that fools bounded-size constant-depth circuits and has 
seeds of size polylogaritmic in the size of the output. Since balanced tables with 
the required parameters are abundant, we infer that there exists a seed s so that 
NW-gen(s) is a balanced table with the required parameters. A balanced table is an 
object of size exponential in n, which implies that the seed s has size polynomial in 
n. Moreover, the Nisan-Wigderson pseudo-random generator has the property that 
each bit of the output can be calculated separately in time polynomial in the length 
of the seed. This implies that the Kolmogorov extractor whose table is NW-gen(s) 
can be computed by a polynomial-sized circuit that has s hard-wired in its circuitry. 



2 Preliminaries 

2.1 Notation and basic facts on Kolmogorov complexity 

The Kolmogorov complexity of a string x is the length of the shortest effective 
description of x. There are several versions of this notion. We use here the plain 
complexity, denoted C(x), and also the conditional plain complexity of a string x 
given a string y, denoted C(x j y), which is the length of the shortest effective 
description of x given y. The formal definitions are as follows. We work over 
the binary alphabet {0,1}. A string is an element of {0,1}*. If x is a string, \x\ 
denotes its length. Let M be a Turing machine that takes two input strings and 
outputs one string. For any strings x and y, define the Kolmogorov complexity of x 
conditioned by y with respect to M, as Cm(x | y) = min{|p| | M(p,y) = x}. There 
is a universal Turing machine U with the following property: For every machine M 
there is a constant cm such that for all x, C\j{x \ y) < Cm{% \ y) + cm- We fix 
such a universal machine U and dropping the subscript, we write C(x \ y) instead 
of C\j(x | y). We also write C(x) instead of C{x | A) (where A is the empty string). 
The randomness rate of a string x is defined as rate(x) = ^rr- If n is a natural 
number, C(n) denotes the Kolmogorov complexity of the binary representation of 
n. For two n-bit strings x and y, the information in x about y is denoted I(x : y) 
and is defined as I(x : y) = C(y \ n) — C(y \ x). 

In this paper, the constant hidden in the O(-) notation only depends on the 
universal Turing machine. 

For all n and k < n, 

2 *-o(i) < | {3 € { 0; iyn | C ( x | n ) < < 2 k . 

Strings x\,X2, ■ ■ ■ ,X). can be encoded in a self-delimiting way (i.e., an encoding 
from which each string can be retrieved) using |zi| + 1#2| + - • • + \xk\ + 2 log |x2| + . • • + 
2 log \xk\ + 0(k) bits. For example, x\ and x<i can be encoded as (bin(\x2 QOlzia^, 
where bin(n) is the binary encoding of the natural number n and, for a string 
u = u\ . . . u m , u is the string uiu% . . . u m u m (i.e., the string u with its bits doubled). 

All the Kolmogorov extractors in this paper are ensembles of functions / = 
(/n)neN of type /„ : {0, l} n x {0, l} k ^ -> {0,l} m ( n ). For readability, we usually 
drop the subscript and the expression "ensemble / : {0, l} n x {0, l} fc -> {0, l} m " is a 
substitute for "ensemble / = (/ n )neN) where for every n, f n : {0, l} n x {0, l} fc ( n ) — » 
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{o,i} m K" 

For any n G N, [n] denotes the set {1,2,..., n}. 

2.2 Approximate counting via polynomial-size constant-depth cir- 
cuits 

In the derandomization argument used in the proof of Theorem 14.61 we need to 
count with constant-depth polynomial-size circuits. Ajtai Ajt93] has shown that 
this can be done with sufficient precision. 

Theorem 2.1 (Ajtai's approximate counting with polynomial size constant- depth 
circuits.) There exists a uniform family of circuits {G n } ne ^, of polynomial size 
and constant depth, such that for every n, for every x € {0, l} n , for every a € 
{0, . . . , n — 1}, and for every e > 0, 

• If the number of 1 's in x is < (1 — e)a, then G n (x, a, 1/e) = 1, 

• // the number of 1 's in x is > (1 + e)a, then G n (x, a, 1/e) =0. 

We do not need the full strength (namely, the uniformity of G n ) of this theorem; the 
required level of accuracy (just e > 0) can be achieved by non-uniform polynomial- 
size circuits of depth d = 3 (with a much easier proof, see |Violl| ) . 

2.3 Pseudo-random generator fooling bounded-size constant-depth 
circuits 

The derandomization in the proof of Theorem l4.6l is done using the Nisan-Wigderson 
pseudo-random generator that "fools" constant-depth circuits [NW94j. Typically, it 
is required that the circuit to be fooled has polynomial size, but the proof works for 
circuits of size 2 log n (where c is any fixed constant). 

Theorem 2.2 (Nisan-Wigderson pseudo random generator.) For every constants 
c and d there exist a constant c' with the following property. There exists a function 
NW-gen : {0, l}°( lo s c ' n ) -> {0, l} n such that for any circuit G of size 2^ n ^ and 
depth d, 

\ P ™ h sm i}o^> n) lG(NW-gen(s)) = 1] - Prob, e{0jl} , [G(z) = 1]| < 1/100. 

Moreover, there is a procedure that on inputs (n,i,s) produces the i-th bit of 
NW-gen(s) in time poly(logn). 

3 Upper bound for the randomness rate of single source 
Kolmogorov extractors with constant nonuniformity 

In this section we show the limitations of what quality of randomness can be ex- 
tracted with a bounded quantity of advice. We obtain this as a consequence of a 
result of Vereshchagin and Vyugin |VV02| . To state their result, let us fix n = length 
of the source, h = number of bits of advice that is allowed, and m = the number of 
extracted bits. Let H = 2 h+1 - 1. 
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Theorem 3.1 ([VV02]) There exists a string x G {0, l} n with C(x) > n — 
//log(2 m + 1) ~ n — Hm such that any string z € {0, l} m with C{z \ x) < h has 
complexity C(z) < h + logn + logm, + O (log logn, log log m). 

The next theorem, a consequence of Theorem 13.11 shows that no Kolmogorov 
extractor for sources with randomness rate a and that uses h bits of advice about 
the source can output strings with randomness rate larger than 1 — (1 — cr)/H. 

Theorem 3.2 Assume that the parameters m, h, a are computable from n and sat- 
isfy the following relations: < a < l,h > 0,0 < m < n, m = w(log n + h). 

Let f : {0, l} n x {0,1}^ — > {0, l} m be a computable ensemble of functions such 
that for every x G {0,1}™ with C(x) > a ■ n, there exists a string a x such that 
C(f(x,a x )) > (l-e)-m. Then e > i^-o(l). 

Proof. Let m' = min(L^^--nJ,m). Note that m' > \J^--m\. Let x be the string 
guaranteed by the Vereshchagin-Vyugin Theorem [XT] for the parameters n,h + c,m' , 
where c is a constant that will be specified later. Note that C(x) > n — H-m' > a-n. 
By assumption there is a string a x such that C(f(x,a x )) > (1 — e)m. Let z be 
the prefix of length m' of f(x,a x ). Note that C(f(x,a x )) < C(z) + (m — m') + 
2 logm + O(l), which implies that C{z) > (1 — e)m — m + m! — 2 logm — O(l) > 
(\-a)m _ ^ m _ 2 j Q g m _ o(\y 

We also have C(z \ x) < \a x \ + c = h + c, for some constant c. It follows 
from Theorem 13.11 that C{z) < h + logn + logm' + 0(log logn, log logm'). So, 
^-jf^ m — em — 2 logm — 0(1) < h + logn + 0(log logn, log logm'), which implies 

that e > i_£ _ ^ogn) = w _ o(1) . | 

4 Single source Kolmogorov extractors with small ad- 
vice 

We move to showing the positive results in Fact [2] and Fact [3] regarding randomness 
extraction with small advice that complement the negative result in Theorem 13.21 
The constructions use the parameters n, ni,m, k, 5 and d. We denote N = 2 n , Ni = 
2 ni ,M = 2 m ,A = 2 s and D = 2 d . We identify in the natural way a function 
E : {0, 1}™ x {0, l}™ 1 -> {0, l} m with an [N] x [Ni] table colored with colors from 
[M]. For A C [M], we say that an (u, v) cell of the table is an A-cell if E(u, v) G A. 
The reader might find helpful to consult the proof plan presented in the Introduction. 
As explained there the notion of a balanced table plays an important role. 

Definition 4.1 A table E : [N] x [Ni] -> [M] is (K,D, A) -balanced if for any 
B C [iV] mtft (SI >K, for any A C [M] ^ > ^, ^ /ioZcfe t/iat 

\A-cells in B x [AMI |A| 
\B\ x Ni M 

The following lemma shows that a balanced table is a good Kolmogorov extractor. 

Lemma 4.2 Lei £ : [N] x [jVi] -> [M] be a (K, D, A) -balanced table and d = 

5 + O(l). Suppose C(E | n) = 0(1) and n\,k,d, and 5 are computable from n. 
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Let (x,y) € [N] x [N{\ be such that C(x \ n) > k + O(l) and C(y \ x) > n 1 . Let 
z = E(x,y). Then C (z \ m) > m — d. 

(Note: 0(1) means that there exist constants, depending only on the universal 
machine, for which the statements hold.) 

Proof. Suppose C(z \ m) < m — d. 

Let A = {w £ {0, l} m | C(w | m) < m — d + 0(1)}, where the constant O(l) is 
chosen so that \A\ > 2 m ~ d . Also note that \A\ < 2 m - d +°W . 

We say that a row v is bad if the number A-cells in the {v} x [iVi] rectangle of 
E is > A • • N\. The number of bad rows is at most K, because the table E is 
(K, D, A)-balanced. Therefore a bad row v is described by the information needed 
to enumerate the bad rows (and this information is derivable from n) and from its 
rank in the enumeration of bad rows. So, if v is bad, C(v \ n) < k + 0(1). 

Since C(x | n) > k + 0(1), it follows that x is good. Therefore, the number of 
4-cells in the {x} x [JVi] rectangle of E is < A ■ ^ • N x = 2 5 - d+ril+ °W . 

Note that, by our assumption, the cell (x, y) is an ^4-cell. Since we can effectively 
enumerate the ^4-cells in the table E, the string y, given x, can be described by the 
rank of (x, y) among the A-cells in the {x} x [N\] rectangle of E. 

So, C(y | x) < 5 — d + n\ + 0(1) and the right hand side is less than n\ for an 
appropriate choice of the constant 0(1) in the relation between d and 5. We obtain 
that C(y | x) < n%, contradiction. | 

The next lemma establishes the parameters for which balanced tables exist. 

Lemma 4.3 Suppose the parameters satisfy the following relations: D = 
0(A), n/ 5 = o(Ni), and M = o(5 ■ K ■ Ni). Then there exists a table 
E : [N] x [N x ] -> [M] that is (K, D, A) -balanced. 

Proof. The proof is by the probabilistic method and is presented in the Appendix. 
I 

We can now prove Fact (2) The formal statement is as follows. 

Theorem 4.4 Parameters: Let m(n) and h(n) be computable functions such that 
m(n) < n for all n and h(n) = a; (log ) . 

There exists a computable function E : {0, l} n x {0, l} /l ( n ) — y {0, l} m(n ) ; such 
that for every x £ {0,1}™ with C(x \ n) > m(n), there exists a x G {0, l}' l ( n ) such 
that C(E(x, a x )\m(n)) > m(n) — o(m(n)). 

Proof. We take 5 = 2(1 5 " , d = 6 + c, where c is the constant from Lemma H~2l 
n\ = h(n). 

By Lemma WM there exists a table E : [N] X [Ni] -> [M] that is (K,D,A)- 
balanced, and by brute force one can build such a table from n. Thus we obtain 
such a table E with C(E \ n) = O(l). We take a x to be a string in {0, l} /l ( n ) 
such that C(a x \ x) > h(n). Using Lemma S2J we obtain that C(E(x,a x ) \ m) > 
m(n) -d = m(n) - 20 5 " (n) - c = m - o(m). I 

Our next goal is to derandomize the construction in Theorem 14.41 As explained 
in the Introduction the key observation is that checking if a table is balanced can 
be done, in an approximate sense, by constant-depth circuits with relatively small 
size. 
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Lemma 4.5 The parameters ni,m, k, d, and 5 are positive integers computable from 
n in polynomial time. We assume k < n,m < k,d < n. 

There exists a circuit G of size j>o\y(N K ) and constant depth such that for any 
table E : [N] x [JVj] -> [M], 

(a) ifG(E) = 1, then E is (K, D, 1 .03A) -balanced, 

(b) if E is (K, D, A) -balanced, then G(E) = 1. 

Proof. Let a = (1/0.99)A • 1/D ■ K ■ N\. Let us fix for the moment a set of 
rows B C [N] of size \B\ = K and a set of colors A C [M] of size \A\ = M/D. Let 
xb,a be a binary string indicating which cells in the B x [N\] rectangle of E are 
^-colored. Formally, xb,a is the string of length K ■ N\, whose (i,j)-th bit is 1 if 
the cell in the rectangle B x [Ni] of E is an ^4-cell and if it is not. 

By Ajtai's Theorem 12. 1\ there exists a polynomial-size constant-depth circuit G' 
(which does not depend on B and A) with a hardwired and such that 

• G'(xb,a) = 1 if the number of A-cells in B x [Ni] is at most (1 — 0.01) • a, and 

• G'(xb,a) = if the number of A-cells infix [Ni] is at least (1 + 0.01) • a. 

Now we describe the circuit G. 

The circuit G on input an encoding of the table E (having length N ■ Ni ■ m) 
computes in constant depth a string xb,a for every B C [N] with \B\ = K and 
for every A C [M] with A = M/D. There are (%)(j$d) = P ol y( NK ) sucn strings 
Xb,a- Each such string xb,a is the input of a copy of G'. The output gates of all 
the copies of G' are connected to an AND gate, which is the output gate. 

If G{E) = 1, then G'(xb,a) = 1 for ah -B's and j4's as above. This implies that 
for all B C [N] with \B\ > K and all A C [M] of size > M/D, the number of A-ceQs 
in the B x [N^ rectangle of E is at most (1 + 0.01)a < (1.03) • A • (1/D) ■ K ■ N x , 
i.e., E is (K,D, 1.03A)-balanced. 

In the other direction, if E is (K, D, A)-balanced then for all B C [N] with 
\B\ = K and for all A C [M] with ^ = M/D, the number of A-ceUs in B x [iVi] is 
at most A • (1/D) • K ■ N x = (1 - 0.01)a, which implies that G(E) = 1. | 

We next prove Fact [3j The formal statement is as follows. 

Theorem 4.6 Parameters: Let m(n) and h(n) be polynomial- computable functions 
such that m(n) < log c n for some constant c and h(n) = cj(log( m ? ^ )). 

T/iere exists a function E : {0, l} n x {0,l} h( - n ^ — > {0, l} m(n ) ; computable by an 
effectively constructible circuit having polynomial size and the following property: 
For every x £ {0, l} n with C(x j n) > m(n) + 0(1), there exists a string a x G 
{0, l} h(n ) such that C(E(x, a x ) \ m(n)) > m — o(m). 

Proof. Let k = m(n),5 = 20 5 " (n) , d = 5 + c + log 1.03 (where c is the constant 
from Lemma l4.2h . and n\ = h{n). 

Let G be the circuit promised by Lemma 14.51 for these parameters. Let dAjtai 
be the depth of the circuit G. Note that the size of G is bounded by poly(A r - ft -) < 
2 logC N (taking into account the bound on m given in the hypothesis). 

Let iV = N ■ N\ ■ m. This is the size of an encoding of a table E : [N] x 

[Nx] -> [M). Let NW-gen : {0, l} 1 ^'^) -> {0, 1}^ be the Nisan-Wigderson pseudo- 
random generator given by Theorem 12.21 that fools circuits of for depth dAjtai and 
size 2 l °z c+2N . 
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The probabilistic argument in Lemma 14.31 can be modified to show that among 
the tables of type E : [N] x [N\] — s> [M] the fraction of those which are (K, D, A)- 
balanced is at least 0.51. Since G accepts all such tables, 

Prob^ e{01 p[G(E) = l]>0.51. 

Since the circuit G has depth equal to d^jtai and size bounded by 2 lo § N , it 
follows that if we replace a random E G {0, 1}^ by NW-gen(s) for a random seed 
s G {0, l} lo § c W , we obtain 

Prob sg{01}logcV) [G(NW-gen(s)) = !] ^ °- 5 - 

We only need the fact that there exists a string s G {0, l} log such that NW-gen(s) 
is a table E : [N] x [N{\ -)• [M] that is (if, D, 1.03A)-balanced. We fix such an s 
that is computable from n (say, the smallest s that has the property) and the 
corresponding table E produced by the Nisan-Wigderson pseudo-random generator 
on seed s. 

Let us consider x G {0, l} n with C{x \ n) > k and a x G {0, l}" 4 with C{a x \ y) > 
n\. Since E is (K,D, 1.03A)-balanced, it follows from Lemma [4.2l that C(E(x,a x ) \ 
m) > m — d = m — o(m). 

Now, let us view E (which is NW-gen(s)) as a function E : {0, l} n x {0, l}™ 1 — > 
{0, l} m . From the properties (i.e., the "Moreover ..." in Theorem 12. 2 \i of the Nisan- 
Wigderson pseudo-random generator, it follows that this function can be computed 
by a polynomial-size circuit which has s hardwired (note that the size of s is poly(n)). 
Since s is also computable from n, one can compute a description of the circuit, i.e., 
the circuit is effectively constructible. I 
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A Appendix 



Proof of Lemma 14.31 . 

The proof is by the probabilistic method. Consider a random function E : 
[N] x [Ni] — > [M]. We evaluate the probability that E fails to be {K, D, A)-balanced. 
Note that if E fails to be (K, D, A)-balanced, then there exists a set B C [N] of size 
exactly K and a set A C [M] of size exactly M/D such that the fraction of A cells 
in the B x [N\] rectangle of E is greater than A • |j4|/M. Let us call this latter event 
5. We show that the probability of S is less than 1. Fix B C [N] of size K and 
A C [M] of size M/D. For a fixed (x,y) £ B x [Nx], Proh[E(x,y) G A] = \A\/M. 
The expected number of A-cells inBx [iVi] is /i = |B| • N\ ■ \A\/M. Let A' = A - 1. 

We use the following version of the Chernoff bound. If X is a sum of independent 
Bernoulli random variables, and the expected value E[X] = [a, then Prob[X > 



(l + A)/i] < e -A(In(A/3))M0 



Using these Chernoff bounds, 



Prob[|A-cells infix [JVi]| > (1 + A')H < e - A '( ln ( A '/3))M. 

The set B can be chosen in (^) < N K ways. The set A can be chosen in 
ways. It follows that the probability of <S is bounded by 

N K ■ (eD) M/D ■ e -A'(ln(A'/3))..Rr-Ar 1 .(i/ J D) j 



( M \ < 
\M/D) - 



which, taking into account the relations between parameters, is less than 1. 



I 




(1 + A) (1 + A) 



M is presented in many 
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